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[57] ABSTRACT 

A timing control arrangement that dynamically con- 
trols .the distribution of timing information in a distrib- 
uted digital communication system. A reference timing 
signal is distributed from a reference master node to all 
other nodes in the system. The distribution is accom- 
plished on a dynamic basis without the use of a central 
control. Each node is connected by links to at least one 
other node and each node receives timing signals from 
all of the links to which it is connected./Eacir node; 
selects one of these signals as its timing reference by 

/scanning the various signals it receives to identify the 
one signal that is applied via a link path that is the "clos- 
est" to the master reference node as indicated by infor- 
mation specifying the number of intermediate nodes 
through which the timing signal has traveled Jrom the 
reference node to reach the receiving node. This dcci-^ 

( sion is based on a set of bits included in the timing signal 
transmitted by each node indicating the number of in- 
termediate nodes between itself and the reference node. 

26 Claims, 12 Drawing Sheets 
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DISTRIBUTED TIMING CONTROL FOR A 
DISTRIBUTED DIGITAL COMMUNICATION 
SYSTEM 

FIELD OF THE INVENTION 

This invention relates to a distributed digital system 
and more particularly to improved facilities for control- 
ling the timing between the nodes of the digital system. 

PROBLEM 

It is necessary that all nodes of a distributed digital 
system, such as a digital s^yitching or transmission sys- 
tem, have precisely synchronized timing so that voice 
and/or data signals can be transmitted error free be- 
tween nodes. One way to achieve this timing precision 
is to transmit timing signals to all nodes from a master 
signal source so that the tuning circuitry at all nodes is 
synchronized to the same signal. For reliability pur- 
poseS} it is necessary that the system timing facilities 
have the capability of maintaining the intemode syn- 
chronization at ail times including circumstances in 
which the master signal source may fail or become 
otherwise unavailable. This reliability has heretofore 
been achieved by the provision of a central controller 
which is connected to all nodes and which exchanges 
messages with all nodes to: 

I (1) Initialize the network and designate the network 
node that is to provide the master signal from which all 
other nodes are to take their timing; 

(2) Monitor the health of all nodes; 

(3) Detect trouble conditions during which one or 
more nodes may receive either no timing signal whatso- 
ever or a timing signal that has become inadequate for 
reliable synchronization; 

/(4) Reconfigure the network timing signal paths 
when trouble is detected so that all nodes continue to 
receive a reliable timing signal. In some cases, particu- 
larly if the trouble is in the master node, this may re- 
quire that axi alternate master source of timing signals be 
specified and activated; and 

(5) Monitor the health of all nodes, via an exchange of 
messages, to ensure that the network reconfiguration is 
successful. 

The above characterized timmg facilities operate 
satisfactory so as to perform their intended functions. 
However, these facilities require the use of a centralized 
intelligent common controller connected to all nodes 
and they are complex, they are expensive, and they 
require that messages be sent continuously between all 
nodes and the central controller. This requirement for 
the continuous exchange of messages is particularly 
disadvantageous since the system is ill equipped to 
transmit and process messages when trouble is detected. 
The reason for this is that the network is already over 
burdened with heavy message flow and message pro- 
cessing chores to route traffic around failed nodes even 
without the additional requirements of reconfiguring 
the flow of timing. Additionally, it is desirable in 
modem networks in which control of routing and other 
f\mctions is distributed, to provide a distributed means 
of controlling timing flow so that a separate centralized 
and architecturally incompatible timing controller is 
not needed. This also allows each node to execute the 
same software. 



SOLUTION 

The above problem is solved and a technical advance 
is achieved by the provisions of improved timing facili- 
5 ties for a multinode digital system. These improved 
timing facilities do not require the use of a centralized 
common controller to: 

(1) Monitor the current health of network timing at 
all nodes; 

10 (2) Detect trouble in the timing of one or more nodes; 
and 

(3) Reconfigure the network timing signal paths 
when trouble is detected so as to maintam the precision 
timing at all nodes. 

15 The above functions are achieved without the use of 
a centralized common controller. Instead, a distributed 
timing control arrangement is utilized wherein each 
node has sufficient intelligence to perform the functions 
characterized in steps 1, 2, and 3 immediately above. 

20 One node is designated as a master in accordance 
with the present invention and this master node pro- 
; vides a timing signal that is propagated to all other 
nodes of the system to provide each node with a timing 
reference of the required precision. The master node ! 

25 generates a timing priority number (hereinafter TPN)J 
say a 1, which is transmitted from the master node,j 
together with the master node timing signal, to every! 
other node to which the master node is directly con-| 
nected by a digital transmission path. Each digital trans- , 

30 mission path in the present disclosure is divided into^ 
frames and time slots in accordance with well known 1 
time division multiplexed techniques. In other words, i 
each node to which the master node is directly con- ' 
nected receives timing signals embedded in the stream 

35 of useful data sent from the master node as well as a 
TPN of 1. Each node other then the master operates in 
such a manner that it increments the lowest TPN it 
receives by one and then sends this incremented TPN to 
each node to which it is directly connected. Thus, each 

40 node to which the master node is directly connected 
receives the TPN of 1 from the master node, it incre- 
ments this TPN of 1 by 1 to form a TPN of 2 and it 
transmits this TPN of 2 to each node to which it is 
directly connected including to the master node. Each 

45 such directly connected node, other then the master, 
that receives the TPN of 2 increments the TPN of 2 by 
1 and sends a TPN of 3 to each node to which it is 
directly connected. 
This process continues so that each node receives a 

30 TPN from one or more other nodes, increments the 
lowest TPN it receives and transmits its incremented 
TPN to each node to which it is directly connected. 

Most nodes in the network are directly connected to 
more than one node and thus will receive a TPN from 

55 each node to which it is connected. In many cases, a 
node may receive TPNs of differing values. In this case, 
the node operates in such a manner that it identifies the 
lowest valued TPN it receives, it increments only this 
TPN and only this incremented TPN is sent,to the other 

60 nod,es. The master node does not perform a TPN incre- 
menting function. Instead the master node always gen- 
erates a TPN of 1 and always sends this TPN of 1 to 
each node to which it is directly connected. The master 
node receives an incremented TPN of 2 back from each 

65 node to which it is directly connected. However, the 
master node does not increment a TPN it receives and 
instead continues to send a TPN of 1 to each node to 
which it is directly connected. 
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The node TPNs are transmitted from a sending node 
to a receiving node over the same path by which data is 
transmitted between the two nodes. Each node, other 
than the master node, operates in such a manner that it 
first identifies the path over which the node receives the 
lowest TPN and then, it derives the timing signal re- 
quired by the node from the data signals transmitted 
over the path that supplies this lowest TPN to the node. 

Each node periodically reads the TPNs it receives 
over the paths to which'it is connected and re-evaluates 
the relative magnitude of the received TPNs to deter- 
mine that the path over which it is presently receiving 
the lowest TPN is the same as the path that priorly 
supplied the lowest TPN and the selected timing signal 
for the node. This re-evaluation is made periodic^y 
and a node takes ho action with respect to the control of 
its timing if no change has taken place regarding which 
path was and still continues to supply the lowest TPN 
to the node. On the other hand, if an analysis of the 
currently received TPNs indicates that the path that is 
now supplying the lowest TPN is different from the 
path that priorly supplied the lowest TPN, then the 
node switches the control of the timing of the node 
from the prior path to the path that presently supplies 
the lowest TPN. This situation occurs when the path 
that priorly supplied the lowest TPN is connected to a 
node that becomes defective and begins to generate 
TPNs of increasing value as subsequently described. 

The above described capabilities whereby each node 
takes its timing from the path that supplies the lowest 

. TPN ensures that each node is controlled by the timing 
signal from the master node over the path that involves 

. the least number of intermediate nodes. 

The node that is to be the master node is designated at 
the time the network is configured and initialized and 
the master node supplies the timing to all nodes by 
means of the timing signals embedded in the data signals 

..that are transmitted from the master node to each node 
to which it is directly connected and from there to all 
other nodes. At the time the network is configured, one 
of the other nodes of the network is designated as the 
backup master node. This backup master functions in 
the same manner as all the ren^aining nodes as long as 
the master node is operative and generates reliable tim- 
ing signals. However, the backup master node supplies 
the network timing in the event the master node fails or 
in the event that the interconnections between the mas- 
ter node and the remainder of the network is severed or 
otherwise becomes unreliable. The backup master node 
is programmed with information specifymg that it is a 
backup master and when it detects that the master nodes 
has failed, it takes charge of network timing by sending 
out a TPN of 1 over each network path by which it is 
directly connected to another node. 

The backup master determines that the master node 
has failed by repeatedly analyzing the TPNs it receives 
and by determining that the TPNs it receives arc incre- 
mented on each successive analysis. The decision that 
the master node has failed is made when the received 
TPNs reach a certain predetermined value. As already 
mentioned, the master node continuously generates a 
TPN of I and transmits this 1 to each node to which it 
is directly connected. This TPN of 1 controls each node 
receiving this TPN in such a manner that the TPN 
generated by each such receiving node is incremented 
by 1 and becomes a two. Thus, each node that is di- 
rectly connected to the master node can only generate 
a TPN of 2 as long as the master node remains operable 



and applies its TPN of I to each node to which it is 
directly connected. Each such directly connected node 
receiving the TPN of 1 cannot possibly generate a TPN 
of other than 2 as long as the master node remains oper- 
5 able since the algorithm by which each node operates is 
that is increments the lowest TPN it receives by 1 and 
transmits this incremented value out to each other node 
to which it is directly connected. 
In an analogous manner, the TPN generated by each 
10 remaining node in the network is constrained and re- 
mains constant as long as the master node and all other 
nodes remain operable and supply valid data including 
constant TPNs. The TPN transmitted by each node is 
determined by the number of intermediate nodes be- 
15 tween a particular receiving node and the master node. 
Let it now be assumed that the master node or the 
paths extending from it to its directly connected nodes 
becomes inoperable or unreliable. The directly con- 
nected nodes now no longer receive a TPN of 1. In this 
20 case, there is no constraint on each such directly con- 
nected node and it no longer is constrained in its genera- 
tion of a TPN by the TPN of 1 received from the master 
node. The directly connected node that is no longer 
receiving a TPN of 1 from the master node now identi- 
25 fies the lowest valued TPN it is presently receiving 
from one of the other nodes to which it is connected and 
it increments this new lowest TPN by I and transmits 
this new TPN to the network nodes to which it is di- 
rectly connected. These other nodes in turn increment 
30 the new TPN they receive and send the newly incre- 
mented TPN back to a first node which, in turn, incre- 
ments the newly received TPN by 1 and sends it back to 
the other node. 
This process continues in such a manner that on each 
35 successive interchange of TPNs between a pair of 
nodes,, the TPN associated with a node is incremented 
by two. The backup master node detects this phenome- 
non as it receives successively higher valued TPNs. 
Finally, when the lowest valued TPN reaches a certain 
40 predetermined number, such as for example 30, the 
backup master determines that the timing of the net- 
work is unstable and it assumes the roll of a master 
node. It docs this by adjusting its circuitry so that it 
immediately begins to send a TPN of 1 to each node to 
45 which it is direcdy connected. This causes the timing of 
the network to become stable and to be controlled by 
the timing signals from the backup master node. 

The backup master node remains in charge of the 
network until it is advised that the master node is once 
50 again operable and at that time the backup master node 
ceases to generate a TPN of 1. The timing of the net- 
work is again brought under control of the master node 
which is now operable and it again supplies a TPN of 1 
to the network nodes to which its is directly connected 
55 in the same manner as priorly described. 

In summary of the above, it can be seen that the 
present invention solves the above discussed problem 
and achieves a technical advance by providing timing 
facilities which permit a multi-node network*s timing to 
60 be accurately controlled at all times either by a master 
node or by a backup node without the necessity of a 
centralized common controller which must continu- 
ously burden the network with the exchange of mes- 
sages between it and the other network nodes. Thus, the 
65 disclosed invention permits the timing of the network to 
be controlled with precision in a manner that does not 
involve the cost or complexity of a centralized control- 
ler. 
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BRIEF DESCRIPTION OF THE DRAWINGS ^[S'^iTv.?"^'^''^! timing back to itself through the 

network, which would create timmg loops and network 

This invention may be better understood by a reading timing instability, 
of the foDowing detailed description thereof with refer- The system timing remains in the state portrayed by 
ence to the following drawings in which: 5 FIG. 1 as long as all elements of the network, including 

FIGS. X and 2 illustrate the network topology of one the master node in path 101 remain operable and supply 
possible network having a plurality of nodes; the required timing signals to node B. Now, let it be 

FIGS. 3, 4, 5 and 6, when arranged and shown in assumed that either master node A or path 101 cncoun- 
FIG. 7, disclose the details of the circuitry provided at ters problems so that node B no longer receives a TPN 
a node; 10 of 1 from node A. In this case, a TPN of 3 is the lowest 

FIG. 8 discloses the protocol used to transmit the valued TPN received by node B. Node B increments 3 
TPNs over the network paths; by 1 and generates a TPN of 4 which is transmitted to 

FIG, 9 discloses the relationship between the system node C as well as node D. Each of nodes C and D 
clock timing and synchronization facilities and the facil- increment the received TPN of 4 and transmit a new 
itics used to control and generate the TPN at each node; 13 TPN of 5 back to node B. Node B, in turn, increments 

FIG. 10 Ulustrates the manner in which the TPN 5 by 1 and returns a new TPN of 6 to nodes C and D. 
generated by node can vary under certain system condi- This process continues with respect to nodes B, C and 
tions; and D so that successively higher valued TPNs are gcncr- 

FIGS. 11 through 17 illustrate in flowchart form the ated. The remaining nodes of the network follow in an 
manner in which the circuitry at each node operates to 20 analogous manner and they also generate successively 
generate a TPN in response to the various system condi- higher valued TPNs. Backup node F receives these 
tions and states that may be encountered by the node. successively higher valued TPNs and when a TPN of a 
r«3T A 11 T\T3 or«D Tirrioivr predetermined value is received such as for example the 

DETAILED DESCRIPTION 3^ ^^^^ p determines that the network timing 

The network topology is shown in FIG. 1 where the 25 is no longer reliable and it assumes control of the net- 
network comprises the plurality of nodes A, B, C, D, E, work timing by changing its operation so that it begins 
F, Qy H and I. The nodes are interconnected by the to generate a TPN of 1 which is transmitted from node 
indicated paths with the numbered arrows. Associated F over paths 107, 106 and 112 to each of nodes C, D and 
with each path is a TPN number that is transmitted over G. 

the path in each direction. The number adjacent to each 30 This condition is portrayed in FIG. 2. The network 
node indicates the TPN that is generated by the node timing soon establishes and the nodes of the network of 
and transmitted on all of its outgoing links. In FIG. 1, FIG. 2 generate and exchange TPNs over the various 
node A is the master node, node F is the backup master network paths as indicated by the numbered arrows and 
and the network conditions portrayed in FIG. 1 indi- the numbers associated with each node on FIG. 2. The 
cate that the master node is functioning properly to 35 backup master remains in control until such time as 
supply timing signals of the required precision to the master node A and/or the path 101 is fixed and again 
rest of the network. Master node A generates a TPN of capable of providing reliable timing signals to node B. 
1 and transmits it to node B as indicated by the arrow The master node A resumes its generation of a TPN of 
pointing in the direction of node B over path 101. Node 1 which is again transmitted to node B. Then, backup 
B receives this TPN of 1, it increments this I by 1 and 40 master node F is advised of this condition and it termi- 
applies a TPN of 2 over paths 101, 102 and 103 to nodes nates its generation of a TPN of 1 . The network is once 
A, D and C, respectively. Node C receives the TPN of again under control of master node A. This switchover 
2, it increments the 2 by 1 and transmits a TPN of 3 to is accomplished in a manner that both master and 
node D over path 104. Node D receives a TPN of 2 backup master are outputting I's and supplying network 
from node B and a TPN of 3 from node C, It also re- 45 timing simultaneously very briefly. This is because the 
ceives a TPN of 4 from each of nodes E and F, The condition of having no master temporarily is more 
TPN of 2 is the lowest TPN received by node D and likely to cause network timing instabiUty and more 
therefore it increments the 2 by 1 and transmits a TPN transmission errors than having two masters teraporar- 
of 3 to each of nodes B, C, E and F. ily. 

Nodes F, G, H and I work m an analogous manner 50 FIGS. 3, 4, 5 and 6. when arranged as shown on FIG. 
and receive the mdicated TPNs. Each such node incre- 7, disclose further details of the equipments comprising 
ments the lowest TPN it receives by I and transmits this any of the nodes A through I of FIG. 1 or 2. Each node 
incremented TPN to each node to which it is directly contains the equipments required to receive and trans- 
connected. Thus, node I receives a TPN of 5 as well as mit information over any of the links or paths to which 
a TPN of 6. Since 5 is the lowest value of the two TPNs, 55 it is connected. This information comprises the TPNs as 
node I increments 5 by 1 and transmits a TPN of 6 to already described as well as data representing the infor- 
each of nodes G and H. mation that is transmitted over the network from a 

The circuitry of each node operates in such a manner sending node to a receiving node via any intermediate 
that the timing of the node is controlled from the signals nodes that may be in the network between the sending 
on the same path that transmits the lowest valued TPN 60 and receiving nodes. 

to the node. Thus, node D receives TPNs of 2, 3, and 4. FIG. 8 discloses the manner in which information is 
Since 2 is the lowest received TPN, node D uses the transmitted over the paths of FIGS. 1 and 2 between 
signals on path 102 to control its timing. In the same nodes. The information is in. timed division multiplexed 
manner, node I uses the signals on path 110 to control its form and is transmitted in eyelid y recurring frames 
timing rather than the signals on path 111. Using this 65 with each frame comprising a plurality of timeslots. In 
algorithm, each node receives its timmg reference signal the partial frame shown on FIG. 8, timeslot 0 contains 
cither directly or indh-ectly from the master over the the TPN information and the other timeslots contain the 
least number of loops possible. Also,, each node is pre- data that is transmitted from a sending to a receiving 
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node. Framing signals divide the signals of each times- 
lot. Timeslot 0 transmits the TPN information in the 
fonn of an eight bit word. The first bit is a spare bit that 
is not presently used. The next bit is a complement bit. 
Five bits are used to represent the TPN. The fmal bit is 
used for parity. The complement bit is used to indicate 
whether the TPN number is sent in a complementary or 
non-complementary form. This capability is used for 
reliabiUty purposes. The TPN can be sent in comple- 



8 



registers and timers which operate under microproces- 
sor control to control the clock system 503 so that the 
equipments of the node 300 operate under control of a 
preferred clock signal in accordance with the present 
invention. The operation of the node 300 and the node 
controller 601 is subsequently described in detail in 
connection with the flowcharts of FIGS. 11 through 17. 

With respect to link interface 301, conductor R of 
link LI carries the signals applied to node 300 and these 
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mentary torm penodjcally to allow faults which cause 10 signals arc applied to receiver 303. These signals are 
one or more bits of the TPN to he stunV at aOnr a l tn Hit/iri/*H intn tiicf^i-^ta. -i • t-t/^ « n 



one or more bits of the TPN to be stuck at a 0 or a 1 to 
be detected quickly. The framing bit is used in a con- 
ventional manner for framing purposes and to permit 
the receiving circuitry to distinguish the bits of succes- 



divided into discrete timeslots as shown in FIG. 8. Re- 
ceiver 303 receives these timeslot signals and applies 
them over its output on path 324 to timeslot extractor 
307. Timeslot extractor 307 extracts the TPN informa- 



sive tuneslots. The TPNs could be transmitted in a 15 tion from the bits of timeslot 0 and applies this informa 



message oriented protocol such as HDLC. but the em 
bodiment shown in FIG. 8 is preferred because less 
processing power and time is required for TPN process- 
ing and transmission because no complex protocol must 
be transmitted 

FIGS. 3, 4, 5 and 6, when arranged or shown in FIG. 
7, disclose the circuitry comprising a typical node of 
FIGS. 1 or 2. The node 300 on FIGS. 3 through 6 
comprise a plurality of line interfaces 301. 302, 401 and 
402eachofwhichterminatesoneof the links or paths to 25 
which the node is connected. The function of the equip- 
ment of a node is to receive information from a con- 
nected link and to apply information to the connected 
link. The bidirectional links on the left side of FIGS. 3 
and 4 are designated LI, L2, L3, and L4. These links 30 
comprise the paths of FIGS. 1 and 2 connected to the 
node portrayed by FIGS. 3 through 6. For example, 
node D of FIG. 1 is connected to the four paths 102, 
104, 105 and 106 and therefore, these four paths corre- 
spond to the bidirectional links designated LI through 35 
L4 on the left side of FIGS. 3 and 4. Only nodes D and 
G of FIGS. 1 and 2 are connected to four paths. The 
remaining nodes are connected to either one path, two 
paths or three paths. For a node connected to only one 



tion over path 328 to incoming TPN register #1 which 
is also designated as element 310. This TPN is applied 
over path 325 from register 310 to the A register 616 
(FIG. 6). of node controller 601. Each of registers 
20 616-619 comprises A and B sections. The signal applied 
to receiver 303 is also extended over path 312 to timing 
recovery element 305 which recovers the clock signal 
embedded in the received data on the R path of link LI. 
This clock signal is extracted and applied over path 326 
to contact 2 of timing source selector switch 504 of 
clock system 503. Recovered clock is also appHed over 
path 323 to receiver 303. Clock maintenance informa- 
tion is applied over path 331 to the receiver mainte- 
nance and interface registers 615. Path 331 comprises a 
maintenance bus and one of its functions is to supply 
register 615 with information indicating whether re- 
ceiver 303 is receiving valid data and whether timing 
recovery element 305 is successful in recovering good 
timing from the signal received by receiver 303 from 
Unk LI. 

Path 330 extends from the clock signal generator of 
the clock system on FIG. 5 to the link interface units on 
FIGS. 3 and 4. On FIG. 3, path 330 connects to timeslot 
extractor 307 and receiver 303. Path 330 carries the 
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path, only one of the links LI dirough L4 would be 40 clock signal that controls the operation of the node and 



connected on FIGS. 3 and 4. Similariy, less than all of 
the links LI through L4 on FIGS. 3 and 4 would be 
used for the other nodes of FIGS. 1 and 2 that are con- 
nected to less than four paths. 

In addition to the link interfaces 301, 302, 401 and 45 
402, each node comprises a digital switching or trans- 
mission system 501, a clock system 503, and a node 
controller 601. The digital switching system or trans- 
mission system 501 comprises the equipments at the 
node that receives the data or information, other than 50 
the TPN information, that is transmitted over the net- 
work between nodes. If the network of FIG. 1 or 2 
represents a telephone network, then the element 501 
would comprise a digital switching system such as a 



therefore, the clock signal on path 330 controls the 
receiver 303 and the timeslot extractor 307. The signal 
from the R lead of link LI is clocked into the receiver 
303 using the recovered clock applied on path 323 and 
the incoming data on path 324 is clocked out of the 
receiver 303 using the clock signals applied on path 330 
from the clock system 503 of FIG, 5. 

The output of receiver 303 is also extended over path 
324 and cable 516 to the input of the digital switching or 
transmission system 501. This path carried all the times- 
lot signals on incoming path R of link LI and applies 
them to the digital switching or transmission system 
501. Thus, system 501 receives all of the timeslot signals 
applied to receiver 303 while incoming TPN register 
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digital central office or a digital PBX wherein each 55 310 receives only the timeslot 0 signals representing the 
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central office or PBX serves a plurality of connected 
subscriber stations. On the other hand, if the network of 
FIGS, 1 or 2 represents a digital transmission system, 
then element 501 would comprise a digital system serv- 
ing a plurality of digital transmission facilities. 60 

Clock system 503 comprises a precision oscillator 505 
and a timing source selector switch 504 which permits 
the timing of equipments at the node to be referenced to 
either the local precision oscillator 505 or to a clock 
signal recovered from the data applied to the node over 65 
any of the connected bidirectional links LI through L4. 
The node controller 601 is a microprocessor controlled 
element having a microprocessor 610 and a plurality of 



TPN that is transmitted over path LI to the node repre- 
sented by the circuitry of FIGS. 3 through 6. 

Digital Switching or Transmission System 501 ap- 
plies the data it wishes to transmit to another node to 
cable 517 and from there to the link interface and link 
that is to extend this data to the other node. Let it be 
assumed that the data is to be transmitted over link LI. 
In this case, the data from system 501 is applied to path 
327 where it is inserted into a signal comprising a series 
of timeslots as on FIG. 8. Timeslot inserter 308 also 
receives from element 311 the TPN that is to be trans- 
mitted in timeslot 0, Inserter 308 combines the timeslot 
data on path 327 with the TPN on path 329 and applies 
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the resultant signal systems to transmitter 306 for trans- 
mission over path T of link LI to the node connected to 
other end of the link. The timeslot inserter receives the 
node clock signal on path 330. The clock signal on path 
330 is also extended to transmitter 306 to control its 5 
operation and this clock signal is embedded in its data 
stream for recovery at the node which terminates it. 
The TPN that is to be transmitted from the node over 
link LI is stored in register 620 and extended over path 
332 to the outgoing TPN register 311 for line interface 10 
301. The outgoing TPN information is inserted into 
register 311 and applied by this register over path 329 to 
timeslot inserter 308. Inserter 308 inserts the outgoing 
TPN into timeslot 0 and inserts the information on path 
327 into the remainder of the timeslots of each frame is 
generated by link interface 331. The timeslots of each 
generated frame are applied from inserter 308 to trans- 
mitter 306 which in turn extends them over path T of 
link LI to the node of the network coimected to the 
other end of link LI. 20 

FIG. 9 portrays the manner in which the clock sys- 
tem 503 interacts with the node controller 601 and, in 
particular, the processor controlled portion of the node 
controller. The node controller 601 applies control 
signals over path 901 to control the setting of timing 25 
source selector switch 504 to determine the source of 
timing signals to which the circuitry that is to control 
the node. This source may either be precision oscillator 
505 internal to the node, or it may be a clock signal that 
is recovered from the data applied to the node over any 30 
one of the links L1-L4 incoming to the node. The clock 
system 503 monitors this operation and returns informa- 
tion over path 902 to the node controller 601 with the 
returend information representing the state of the sys- 
tem 503 under various conditions that may be encoun- 35 
tered. In most instances, the information on path 902 
will require no change as to the clock signal that is to 
control the node. On the other hand, under certain 
system conditios or states, the returned information may 
cause the node controller to decide that the signal 40 
source currently controlhng the node timing is unreli- 
able and that therefore, the timing control of the node 
should be switched to a different source of clock sig- 
nals. The node controller 601 constantly monitors 
TPNs coming into the node and applies control signals 45 
on path 901 to control which internal or external source 
of timing clock system 503 selects as its timing reference 
signal based on current and recent TPN information 
and the maintenance information received on path 902. 
The manner in which this is accomplished is described 30 
in detail in the flowcharts of FIGS. 11 through 17. 

FIG. 10 portrays the manner in which the TPN gen- 
erated by each node normally remains steady but may 
vary under certain conditions. The first TPN received 
by a node is shown on the top line of FIG. 9 and is 55 
designated TPN #1, The next line down is designated 
as TPN #2 and so on for the third and fourth hnes 
which are designated TPN #3 and TPN #4. The bot- 
tom line is a time line having three different states. The 
left most state represents a steady state in which the 60 
node controller has analyzed the four incoming TPNs 
and has selected the lowest one of them to control the 
timing of the node. The second state of the system that 
may be encountered is termed "Select Precision Oscilla- 
tor". During this state, the TPN received on each path 65 
fluctuates as indicated. This represents an unstable tim- 
ing state of the system and, if the timing becomes suffi- 
ciently unstable, the node controller selects the preci- 
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sion oscillator 505 of the node to be the source of timing 
for the node for the time being. The next state of the 
system is entitled at the top "TPN Steady at Possibly 
New Values". The value of the TPN received on each 
path is steady as indicated by the horizontal line associ- 
ated with each TPN and once again the node controller 
selects the lowest valued TPN to identify the path that 
provides the source of clock signals for the node. In this 
example, TPN #4 remains the lowest after the fluctua- 
tions and the recovered timmg signal from the link 
supplying TPN #4 would again be selected as the 
source of timing for the node. This switch to a new 
source of timing signals is made a finite interval of time 
after conditions stabilize as is subsequently described in 
connection with the flowcharts of FIGS. 11 through 17. 

The following describes the operation of the system 
under various conditios with reference to the flow- 
charts of FIGS. 11 through 17. The process begins with 
element 1101 on FIG. 11 when the system is first acti- 
vated. The system is activated at element 1101 and the 
process continues to element 614 where the system 
reads the contents of the alarm register 614; This regis- 
ter stores the alarms representing the various conditions 
that may be encountered by the system. The process 
continues to element 1115 where it is determined 
whether or not alarm register 614 is set to a state repre- 
senting an alarm. If the answer is no, the process ad- 
vances over path 1123 to element 1117. If the answer is 
yes, the process advances over path 1126 to element 
1116 in which the system clean the alarms and alarm 
register 614. The process then advances over path 1127 
to element 1117 in connection with which the B portion 
of the clear timing source select register 602 is initial- 
ized. This register 602 has an A portion and a B portion 
as indicated and the contents of the A portion of the 
register control the setting of the switch 504. In other 
words, the A portion of register 602 contains informa- 
tion identifying the timing source that presently con- 
trols the timing of the node. This register has an A and 
a B portion whose contents are sometimes compared as 
is subsequently described. 

FIG. 11 represents an initialization operation and 
during this initialization, element 1117 clears the B por- 
tion of register 602. The process continues over path 
1128 to element 1118 in which the B portions and the 
out-of-service bit of the A portions of the TPN registers 
616, 617, 618 and 619 are cleared. The process continues 
to element 1119 in which, the TPN stable timer 607 is 
reset. The TPN stable timer 607 and the subsequently 
described PLL timer 606 and read TPN 605 timer re- 
main m the reset state until explicity started. The pro- 
cess then contines to element 1120 in which the system 
resets the PLL timer 606. The process continues to 
element 1121 in which the system then enters the preci- 
sion oscillator code into the timing source select register 
602. This code of the precision oscillator is entered into 
the A portion of register 602. The entry of this code into 
the A portion of register 602 controls the operation of 
switch 504 via controller 508 so that switch contact W 
engages contact 1 of the switch to connect the output of 
the precision oscillator 505 to the input of the phase 
lock loop 506 via path 513. This switches the control of 
the timing circuitry of the node to the output of the 
precision oscillator 505. During initialization and some 
subsequently described failure states, the precision os- 
cillator is selected as the source of timing for all nodes 
so that the errors rate on the links will be as small as 
possible and the system will be able to transmit enough 
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valid TPNs to progress to a synchronized state as subse- 
quently described. 

The process of FIG. 11 continues to element 1122 in 
which the system clears the access bit portion 612 of the 
external access mode register 611. The state of the ac- 5 
cess bit is, as subsequently described, determines 
whether the node controller examines and responds to 
the information in the external access node register 611. 
• It analyzes the contents of register 611 is the access bit 
612 is set to a 1. It disregards the state of the register 611 10 
if the access bit 612 is a zero. As is subsequently de- 
scribed, the external access mode register 611 is used to 
override the state of switch 626 and to cause the clock 
system 503 to respond to the information in register 611 
rather than respond to the information that is normally 15 
provided by the present position of switch 626 and the 
TPN information. A node normally operates as a master 
node if switch 626 is set to position 624 and as a backup 
master if the switch is set to position 625. The node 
operates as neither a master or backup master if switch 20 
626 is set to position 621. 

The process proceeds over path 1133 to element 1134 
in which the mode status register 608 is set to the link 
mode. The term link mode means that the node does 
not, for the time being, T}perate as a soiu'ce of timing 25 
signals, but, instead, is to have its timing circuitry con- 
trolled by a timing signal received from another node. 
The above described functions on FIG. 11 initialize the 
system and prepare it for the following described. 

The process next extends from P on the bottom of 30 
FIG. 11 to P on the top of FIG. 12, The process pro- 
ceeds over path 1201 to element 1202. This is a decision 
element in which a determination is made as to whether 
the mode status register 608 is in its link mode state or in 
its source mode state. The mode status register 608 is 35 
every node is set at the present time to its link mode 
state since this was done in element 1134 when the 
. system was initialized. Therefore, the process extends 
. .over path 1203 to element 1205 which determines 
. whether or not the master register 622 is presently set 40 
by switch 626, Let it be assumed for the time being that 
the node that is being described with reference to the 
flowchart of FIG. 12 is the master node. Therefore, the 
master register 622 is set and the process continues over 
path 1206 to element 1207 which sets the mode status 45 
register 608 to its source mode status. This indicates that 
this mode is to provide the timing signals for the net- 
work. In other words, it is to be the master node. The 
process continues over path 1208 to element 1209 in 
which the node TPN register 620 is set to 1. The pro- 50 
cess continues over path 1210 to element 1211 which 
causes a TPN of one to be transmitted out from this 
node to all other nodes to which this node is connected 
directly via one of its links L1-L4. The process next 
advances to element 1225 which enters the code of 55 
precision oscillator 505 into the A portion of timing 
source select register 602. This causes switch 504 to 
connect the output of oscillator 505 to phase lock loop 
506. Clock signal generator 507 then supplies a clock to 
system 501 and the link interfaces 301, 302, 401 and 402 60 
which embed the clock in the data they transmit to 
other nodes. This node is therefore now in charge of the 
network timing. 

The process now advances to the circle P at the top 
of FIG. 12. 65 

The process extends through element 1202 to the 
Source Mode output of the element since the presently 
described node is operating as a master. The process 
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now advances over path 1204 to element 1213. The 
backup register 623 is not set for this node and therefore 
the process advances over path 1214 to element 1205 
which determines that the master register 622 is set. The 
process then advances through elements 1207, 1209, 
1211 and 1225 whose functions have already been de- 
scribed. 

The process repeats, as already described, through 
elements 1202, 1213, 1205, 1207, 1209, 1211, and 1225 
which continues to cause a TPN of 1 to be extended out 
over all links of this node to the nodes connected to the 
other end of links L1-L4. The master node continues 
sequencing through the same elements as already de- 
scribed as long as it remains in an operable state and in 
charge of the control of the timing of the network. 

The following paragraph describes the process fol- 
lowed on FIG. 12 for a node that is not the master node. 
All such nodes are referred to as a link node. The pro- 
cess begins on element P at the top of FIG. 12 and 
extends over path 1201 to element 1202 which deter- 
mines that the node is a link node and extends the pro- 
cess of over path 1203 to element 1205. Switch 626 of 
this node is not set to its master position and therefore 
the master register 622 of this node is not set. The pro- 
cess continues from the NO output of element 1205 and 
over path 1222 to element 1223 which is designated as 
an A within a circle. 

The process advances to element A on the top of 
FIG. 13. The process extends from element A within 
the circle to element 1301 which causes the mode status 
register 608 to be set to the link mode. This causes the 
node controller of the node to operate in its link mode 
in which the timing of the node is controlled by signals 
received over the links L1-L4 connected to the node. 
The process then extends over path 1302 to element 
1303 which causes the A half of the TPN registers 616 
through 619 to be read. These registers at this time 
contain the TPNs that are being received over the in- 
coming links L1-L4 from other nodes. 

The process then extends over path 1304 to element 
1305 which determines whether the complement bit of 
the timcslot per FIG. 8 is in a set state. If it is not in a set 
state, the process extends over path 1306 to element 
1310 which determines the validity of the received 
TPN. If the complement bit is in a set state, the process 
extends from element 1305 and over path 1307 to ele- 
ment 1308 which complements the received TPN to 
derive the true TPN. The process then extends over 
path 1309 to element 1310 which determines the valid- 
ity of the received TPN. If the received TPN is valid, 
the process extends over path 1312 to element 1313 
which causes a validity bit of 1 to be written into the A 
section of the TPN register associated with this TPN. 
This is one of registers 616 through 619, The particular 
TPN register that is written is that which is associated 
with the link on which the TPN now being analyzed is 
received. The process then extends over path 1315 to 
element 1317. 

If element 1310 determines that the received TPN is 
not valid, the process extends over path 1311 to element 
1314 which causes the analyzed TPN to be invalidated. 
A TPN is invalid if it has bad parity, if it is out of the 
valid range of TPNs, or if its associated out of service 
bit is set to a I as subsequently described. This is accom- 
plished by writing a validity bit equal to 0 in the A 
section of the TPN register associated with this re- 
ceived TPN. The process then extends over path 1316 
to element 1317 which causes the previously described 
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four steps, beginmng with element 1305, to be repeated TPN generated by a node is used in the present inven- 

for each of the remaining three received TPNs. Finally, tion to indicate a trouble condition, 

after all four received TPNs have been deteremined to Let it be assumed that a TPN of 30 generated by a 

be valid or invalid, as the case may be, and the appropri- node indicates that the node is not receiving valid tim- 

ate results entered into the associated TPN registers, the 5 ing information from another node in the network. This 

process extends over path 1319 to element 1320 which is determination is made by element 1504. If the answer to 

designated as a B within a circle. the decision of element 1504 is no, then the node is 

The process next extends from the bottom of FIG. 13 receiving good timing signals and the process extends 

to element B on the top of FIG. 14. At this time, the over path 1505 to element 1516 which causes the node 
node controller dOl contains in the A section of its four 10 TPN in register 620 to be outputted to the outgoing 

TPN registers 606 through 619 the four TPNs that are TPN registers 311, etc. which, in turn, transmit the 

received on the four links L1-L4 incoming to this node. o^er their associated Unks. The process then 

This statement presupposes that this node is connected extends to element 1517 which is a G within a circle and 

to four links and therefore four TPNs are received. If there to element G on top of FIG. 16. FIG. 16 is 

this is not the case, then a fewer number of TPNs are subsequently described. 

received on the links that are connected to the node and . element 1504 determines that the value of the TPN 

these TPNs are stored m the appropriate ones of regis- ^ '^^^ register 620 has reached a maximum 

ters 616 through 619. value, such as for example 30, the process then extends 

The process extends on FIG. 14 from element B to ^^^^ P*^^ *o element 1507. This element deter- 

clcmcnt 1401 which causes the node conUoIler 601 to """^ whether or not the node being described is a 

identify the one of the TPN registers 616 through 619 backup master node. If it is not, then the process extends 

that contains the lowest incoming TPN. This lowest over path 1509 and the node TPN in register 620 is 

TPN is identified and the process extends over path transferred to the hnc interfaces and from there over the 

1402 to element 1403 which causes the node TPN regis- r^pective links to the other network nodes. On the 

ter 520 to be set to a value that is equal to the lowest ^^^^ /'^^^ ^^^"^ described is a backup 

incoming TPN incremented by one. This number in "^f^^tf ""^^^ ^ ^° ^^^"^^""^ ^^^^ 

register 620 becomes the TPN of the node. The process ^ ^ '"^^P^l ^? ^PP^^ °"tP"* 

extends over path 1404 to element 1405 which causes P^^^!^ ?5^**^5 the phase 

the TPN in register 620 to be transmitted over path 332 ,^ S.iT, , J^^ f ^^"^ ^^^^''^J?? 

to the outgoing TPN register in each of the link inter- ^^^l/^^ the clock signal generator 507 to 

faces such as 301. This is register 311 for interface 301. ^"^'^^ P^^^*"^'^ 

'Si tr^t^^J^rS^^r^^^ P^°^« '^'"^ ^^^^"d^ ^l<^^^"t 1510 which sets 

?hi; ™ tht TpTJ r.^^^^^^ ? Z ^ "^^'^ ^8 ^^<^^ t° indicate 

r^!r.t^rt^^^^^^^^ 35 that this mode is now generating the timing signals 

S n^t^vT^^^^^^^^ 7% \ I'^^A^'T ^^^^^ ^^'^^^l ^^t^^^l^' The process nexTex- 

T^.u X r l^' P^'^ 1511 to elemetn 1512 which causes a 

network node comiected to the other end of the Imk. ^PNof 1 to be entered into node TPN register 620. The 

idS^lhl^Tn,^ T\ ^^f.u^. '° P^^^'*^ ^^t«^ds <>^^' P^th 1513 to element 1514 

l^J^lVT^ u^'kTT.u^ t^T^. ""l ^^^^ ^^^^ "^de TPN of I to be transmitted out 

[wfc *!^°"«^/l^ B portion half of via the line interfaces ofFIGS. 3 and 4 to the associated 

these same registers. As is subsequently descnbed, this ij^ks and from there to the network nodes to which 

pennits a companson to be made between a pnorly these links arc connected. The process then extends 

received and a presently reco^ved TPN for each regis- over path 15 to element 1135 which is designated as a P 

T^^nS"'' u^u ° proceeds over path 140S to ele- 45 within a circle and is also shown on the top of FIG. 12. 

ment 1409 which causes the node TPN to be transmit- m partial summary, the process shown on FIG. 15 

ted out from the outgomg TPN reg^ter of each Imk has determined that the backup master node is receiving 

mterface out over the associated Imk to the network invalid timing information and has placed it in charge of 

node connected to the other end of the Imk. The pro- the network for the time being by cuasing it to begin 

cess then proceeds over path 1411 to element 1412 50 outputting a TPN of 1 to the nodes to which it is con- 

which IS designated F within a circle and from there to nected. 

the top of FIG. 15. p^om element P on the bottom of FIG. 15, the pro- 

On the top of FIG. 15, the process extends over path cess now returns to element P on the top of FIG. 12 

1501 to element 1503 which reads the present content of where the process enters element 1202 which deter- 

aU TPN A registers 616-619. The process then extends 55 mines that the mode status register 608 is set to the 

to element 1504 which determines whether or not the source mode and therefore outputs the process over 

TPN m the node TPN register 620 equals its maximum path 1204 to element 1213. From there, since the backup 

value. What is meant by maximum value is that there is register 623 is set, the process extends over path 1215 to 

a range of acceptable TPN values that the TPNs may element 1216. 

assume without representing a network trouble condi- 60 Let it be assumed for the time being that the access bit 

tion. As ah-eady mentioned, the master node generates a 612 is not set to a 1 and therefore, the process extends 

TPN of 1 and each other node in the network generates over path 1218 to elemetn 1207 which causes the mode 

a TPN which is dependent upon the number of nodes it of the status register 608 to be set to the source mode, 

is removed from the master node. It has also been men- Since this register is already in this mode, no further 

tioned how the TPNs generated by the various nodes 65 action is taken by this element. The process then ex- 

incrcase if the master becomes inoperable and no longer tends to element 1209 which sets the mode TPN regis- 

supplies a TPN of 1 to each node which it is directly tcr 620 to a 1 and then outputs this 1 on all outgoing 

connected. This continued increase in the value of the links of the node in element 1211. Element 1225 selects 
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the precbion oscillator 505 so that it is in charge of the timing function. The purpose of the read TPN timer is 

timing as already described. The process then extends to limit the rate at which the reading operation of the 
over path 1212 then loops through the same sequence of TPN take place. The timer prevents this operation from 

elements as long as the access bit register 612 is not set occurring to frequently by permitting it to occur only 

Finally, after a period of time the maintenance per- 3 when the timer times out. The process now extends 

sonnel will fix the problems in the master node and this over path 1613 to element 1614 which is also designated 

node will then once again take charge of the network. H within a circle. From there, the process continues to 

This is done by transmitting an appropriate message in element H on FIG. 17. 

one of the timeslots to this node. This message is entered On FIG. 17, the process extends from element H to 

into register 502 of system 501 and is transferred from 10 element 1701 which compares the A and B sections of 

there over path 604 where its contents are entered into the TPN register 616-619 pair of pairs having the low- 

the external access mode register 611 with the access bit est TPN. If a single A and B register pair has the lowest 

612 being set. The process then extends from element P TPN then only its A and B sections are compared. If 

on FIG. 12, through element 1202 to element 1213 two or more such pairs have the lowest TPN, then the 

where the backup register 623 is still set since the cur- 15 A and B section of each pair having the same lowest 

rently described mode is a backup master mode. From TPN is compared. 

there the process extends to element 1216. At this time, The process then extends over path 1702 to element 
the external access bit 612 is a 1 since a message has 1703 which determines by a comparison of the A and B 
been received by clement 502 and entered into external section of the various one or more TPN register pairs 
access mode register 611. Therefore the process now 20 whether there has been any changes in the lowest TPN 
extends over path 1217 to elemetn 1219. Element 1219 recently received. If the answer is no, the process ex- 
now analyzes the message in the external access mode tends over path 1705 to element 1706 which determines 
register 611 to determine its contents. Since it is now whether the TPN stable timer 607 has been started. This 
desired that the master node resume control of the sys- timer is normally in a timed out state corresponding to 
tem, the message presently in register 611 specifies that 25 the state of the system in which no change in the lowest 
the node is to once again operate as a link node. This TPN has occurred for a period greater than its timeout 
means that it is to receive timing signals from other period. It is reset when a change is detected in the low- 
nodes and to cease operating as a master node. There- est received TPN. A new link cannot be selected as a 
fore, the process now extends over path 1220 to element timing source until this timer times out. This timer 
A and there to FIG. 13 which enters the process already 30 thereby affectively limits the frequency at which new 
described. links can be selected as new timing sources. 

With reference to element 1219, if the message writ- If element 1706 determines that the TPN stable timer 

ten into register 611 specifies that it is to contmue oper- 607 has not been started, the process extends to element 

ating as a source node, then the process extends over 1710 where the timer is started. If the answer is yes in 

path 1221 to element 1207 and die process continues 35 element 1706, the process extends over path 1707 to 

using elements 1209, 1211 and 1225, etc. as akeady element 1712 which determines whether or not this 

described. timer has timed out. Thus, both paths 1707 and 1711 

At this time, the master node is once again in charge extend the process to element 1712 which now dcter- 

of the system, it is generating a TPN of one and the mines whether or not the TPN stable timer 607 is timed 

timing signals it transmits out over its links are the tim- 40 out 

; ing signals that control timing of all other network If the timer has timed out then the process extends 

nodes as already described. over path 1713 to element 1714 which determines 

Now let it be assumed with reference to FIG. IS that whether or not all valid TPNs are at their maximum 

element 1504 determines that the node TPn is within an value. It has been prioriy mentioned that the maximum 

acceptible value and so the process extends over path 45 value for a TPN is 30 and that a TPN of 30 represents 

1505 to element 1516 and from there to element G on a trouble condition in which the node may not be prcs- 

FIG. 16. On FIG. 16, the process extends from element ently receiving valid timing signals. Therefore, element 

G to element 1601 which reads the contents of the TPN 714 determines whether or not a TPN of 30 is now 

A registers 616 through 619. The process then extends being received on all links. It analyzes the contents of 

to element 1602 which fmds the TPN A register having 50 the A registers 616 through 619 to make this determina- 

the lowest TPN. The process then extends over path tion. If the answer is no, the process extends over path 

1615 to element 1603 which determines whether or not 1715 to element 1716. Element 1716 cuases the code of 

more than one A register has the same lowest TPN the prioriy designated link with the lowest TPN (see 

value. If the answer is no then the process extends over FIG. 16) to be written into the A port of the timing 

path 1605 to elemetn 1606. In this element the micro- 53 source select register 602. This register controls the 

processor 610 operates internally to preliminarily desig- switch controller 508 to set timing source selector 

nate the link associated with the lowest TPN. This link switch 504 to the position associated with the desig- 

is to be the preferred timing source for the node. nated link having the lowest TPN. 

With reference to element 1603. if the answer is yes, The process now extends over path 1717 to element 

the process extends over path 1604 to element 1607 60 1718 which determines whether or not the contents of 

wherein the processor 610 operates internally to desig- the A section of the timing source select register 602 

nate one of the links having the lowest TPN as the link equals the contents in the B section of the register. If the 

that is to be the preferred timing source for the node. answer is yes, this means that a new link has not been 

The process.then extends either over path 1608 from selected and the process continues over path 1719 to 

element 1606 or over path 1609 from element 1607 to 65 element 1720 which rests the PLL timer 606. This timer 

element 1610 which causes the read TPN timer 605 to is normally in a timed out state and only then can one 

be reset. The process then extends over path 1611 to read the clock maintenance interface register 603. This 

element 1612 which starts the read TPN timer and its register interacts with the phase lock loop 506 to detcr- 
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mine whether or not the phase lock loop is currently With reference to clement 1712 let it now be assumed 

responding to a valid timing signal so as to properly that the TPN stable timer 607 has not timed out. In this 

control the clock signal generator 507 to generate a case the process extends over path 1739 to element 1726 

valid timing signal for the node. This timer gives the and to the processes already described. In this case, the 

phase lock loop 506 time to lock onto a new signal. This 5 TPN A registers 616 to 619 containing the lowest TPN 

timer also gels reset after there is a change with respect are stable, but the TPN stable timer 607 has not timed 

to the link that is to control the timing of the node. out, and a new link is not selected as a timing reference 

The process now extends over path 1721 to element because there is till a high probability of further TPN 

1722 which starts the PLL timer 606. The process then changes. 

extends over path 1723 to element 1724 which transfers 10 With respect to element 1714 let it be assumed that all 

the contents of the A section of the TPN registers 616 TPNs are at their maximum value. The process contin- 

through 619 to the B section of these registers. <*ver path 1741 to element 1740 which enters the 

The process continues over path 1725 to element precision oscillator into the timing source 
1726 which causes the contents of the read TPN timer register 602. ,The process then continues over path 1742 
605 to be read. The timer is read and then the process elements beginning with 1726 whose function has 
extends over path 1726A to clement 1728 which deter- already been described. In this case all valid TPNs in 
mines whether or not the timer has timed out. If it ^ registers 616 to 619 are at the maximum 
hasn't, the process loops over path 1727 and element value, there could be no valid source of timing from any 
1726 and path 1726A and then once again 1728 until the ^^^^^ ^ "° operating master node in the net- 
timer does time out This tuner prevents the system work. Therefore the node timing will be generated from 
from reading the TPNs too often. precsion oscillator until this condition ceases. 

Finally, a determination is made that the timer has With regard to element 1718 now let it be assumed 

timed out and the process continues over path 1729 to , contents of the A section of the timing source 

element 1730 which moves the contents of the TPN A ^ . ^2 do not match the contents of the B 

registers to the TPN B registers. These are the register ^^^^^ ^^^^^^ ^^^^^^y "idicating that the source 

pairs 616 through 619. The process then continues over °[ ^« r ^^^^ J"^"^^^"^ J""** 

path 1731 to element F and from there back to element J^^f^'/" ^^.""^ f continues over path 

F on FIG, 15 where the processes already described are .l^^or'^T^ ^ xil"^ which determines whether or not 

entered r / the timer 606 has timed out. If it is not timed out, 

Briefly, the process of FIG. 15 outputs the current clock maintenance register 603 cannot be read 

nodeTPNifthenodeTPNisnotatits^iaximum value. ^V^Z ?\%^^ ^^^"1 "7 extends over path 1745 

It causes the node to become a source node if the node LhTS^I^ Jpt ? t"" '^-Sii^f '^'''"^^ u"*"" 

T-nxT 1 J J . . ■ , scnoed. When the PLL timer 606 tunes out, this indi- 

ThefoUowingparagraphsdescribethcpathsonFIG. " lri,?onS™rW^^^^^ 

17 f>iaf Ka*/« „^/u ° Jii^^u. u A r\ ' " ^ opcratmg properly and that the frequency of its 

ll^l l '^"'T^- P"'* ^S*^' input U not within iu lock-in range. Then the process 

TPM «St 1"?" IJ^T" ^^t-'-ds over path 1746 to element 1747 which r<^ds the 

°f »he clock maintenance register «03. The 

" '^l '° °f » » whence phase lock 

^fcll • • . ^Sf. « -^f 1°°P « ^P^^8 properly and 0 when it is not 

7^T^ I- extends over path 1704 to element agister's contents are a 1, then the process continues 
1732 which determine whether or not the precision over path 1750 to element 1724 whose function has 
oscdlatorSOS issdectedan^^^^^ « already been described. This indicates that the phase 
ter «!02 to adjust switch 504 Ae answer is yes, the locked loop 506 b operating properly and is locked to 
proce^ont.nu« over path 1^3 and the TPN stable and hence deriving timing from its mp»t signal from 
tmicr 607 IS reset in element 1737. If the answer of ele- timing source selector switch 504. Hence no corrective 
moit 1732 IS no, then the precKion oscillator code is action is required. On the other hand, if element 1749 
entered mto the A portion of the tmung source select 50 determines that the clock maintenance register contains 
register «« aid Ae process contmues over path 1736 to a 0, this means that the phase lock loop 506 is not receiv- 
element 1737. The resetting of the TPN stable timer hig » good timing signal. In this case, the process ex- 
prevents a new hnk from bcmg selected until this timer tends over path 1751 to element 1752 which causes the 
tunes out. The process now continues over path 1738 to precision oscUlator code to be entered into the A sec- 
element 1726 which reads the contents of the read TPN 55 tion of the timing source select register 602. This pre- 
timer 605 and then element 172* determines whether or pares the node to operate under control of its precision 
not the read TPN timer 605 is timed out. The function oscillator 505. The process then extends over path 1753 
of element 1730 has already been described. to element 1754 which writes an out of service bit of 1 
Thus, m partial summary, the process just described next to the TPN of the selected link in the A section of 
begmnmg with elements 1703 and 1732 caused teh pre- 60 the TPN register 616 through 619 associated with the 
cision escalator to be selected. A new link wUl be se- selected link. This out of service bit of 1 in this register 
lected as the nodes source of timing after the TPN indicates that the phase lock loop 506 is having diffi- 
fluctuations cease and the ssytem returns to a stable culty following the timing signal provided by the se- 
state as indicated by steady TPNs. However, a new link lected link. This determination was made when it was 
cannot be selected until after the TPN stable timer times 63 determined that the clock maintenance register 603 
out and we can be relatively certain that there wUl be no contains a 0 rather than I. 

more fluctuations in the TPN A registers 616 to 619 A 0 in register 603 indicates that the phase lock loop 

containing the lowest TPN. S06 cannot successfuUy follow the timing signal to 
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which the phase lock loop is currently connected by 
switch 504. That is the reason that element 1752 entered 
the precision oscillator code in the timing source select 
register 602 to cause the swtich 504 to connect the 
phase lock loop 506 to the output of the precision oscil- ^ 
lator 505 and to disconnect it from the link providing 
the unreliable timing signal. Therefore, element 1754 
writes an out of service bit of 1 into the A section of the 
TPN registers associated with this link. This out of 
service bit of 1 advises the processor 610 that this TPN 10 
for this link in this register should be disregarded until 
the system is reinitialized starting at element 1101 of 
FIG. 11. The reason for doing this is to prevent the 
clock system from attempting to synchomize to this 
defective link on subsequent reading of the TPNs, This *^ 
condition persists until the out of service bit for this 
regbter is switched from a 1 to a 0 to indicate that the 
associated link may once again by providing a reliable 
timing signal. 

The process next continues from element 1754 and 
over path 1755 to element 1756. This element causes an 
entry to be made in alarm register 614 to activate the 
alarm LED's 613. These provide an alarm to the main- 
tenance personnel indicating that one of the incoming 
links is providing an unreliable timing signal. The main- 
tcnance personnel then take the appropriate action re- 
quired to remedy the situation. 

The process next advances over path 1757 to element 
1724. The element causes the current contents of the A 
section of the timing source select register written into 
the B section of the register. 

The process next advances over path 1725 to element 
605 and the elements subsequent thereto. This portion 
of the flow chart has already been described. 3* 

On FIG. 6, conductors 511, 510, 604, 331, 325, 335, 
425, 435 and 332, respectively are shown as being di- 
reclty connected to registers 603, 602, 611, 615, 616, 
617, 618, 619, and 620, respectively for ease of under- 
standing of the relationship between these conductors 4^ 
and registers. In practice, since node controller 601 is 
processor controlled, these registers would be part of 
the memory of microprocessor 610 and these conduc- 
tors would be connected to I/O facilities of the micro- 
processor. 45 

While a specific embodiment of the invention has 
been disclosed, variations within the scope of the ap- 
pended claims are possible and are contemplated. There 
is no intention of limitation to what is contained in the 
abstract or the exact disclosure as herem presented. The 50 
above-described arrangements are only iUustrative of 
the application of the principles of the invention. It is to 
be expected that other arrangements will be devised by 
those skilled in the art without departing from the spirit 
and the scope of the invention. In particular, an embodi- 55 
ment where the precision oscillator is replaced by a 
combination of a precision oscillator and one or more 
timing reference signals of origin outside the presently 
described network of nodes is contemplated. 

What is claimed is: 60 

1. A method of controlling the timing of each node in 
a multinode system whose nodes are interconnected by 
links and wherein a clock source in one of said nodes 
provides a reference clock signal source for controlling 
the timing of all other nodes in said system, said method 65 
comprising the steps of: 

(1) operating one of said nodes as a master node to 
control the timing of all nodes of said system, 
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(2) transmitting a clock signal between each pair of 
said nodes directly interconnected by any one of 
said links, 

(3) including timing priority number (TPN) informa- 
tion in the clock signal transmitted between each 
pair of directly connected nodes with said informa- 
tion representing the number of intermediate nodes 
said clock signal has encountered in traveling from 
said master node to the node receiving said clock 
signal, 

(4) operating each clock signal receiving node, other 
than said master node, to analyze the TPN informa- 
tion it receives on all links to which it is connected 
to identify as a preferred clock signal the clock 
signal that has encountered the least number of 
intermediate nodes in traveling &om said master 
node to reach said clock signal receiving node, 

(5) modifying the received TPN from the intermedi- 
ate node generating said preferred clock signal, 

(6) transmitting the modified TPN to all intermediate 
nodes, and 

(7) operating each clock signal receiving node, other 
than said master node, to select said preferred 
clock signal as a reference timing source for said 
receiving node. 

2. The method of claim 1 in combination with the 
additional steps of: 

(1) operating each node, other than said master node, 
to repeatedly scan said TPN information it re- 
ceives, 

(2) detecting when the received TPN information 
associated with said preferred clock signal changes 
on successive scans, and 

(3) switching the timing control of each node that 
detects a change from said preferred clock signal to 
a signal from a local oscillator internal to said last 
named node upon the detection of said change. 

3. The method of claim 2 in combination with the 
additional steps of: 

(1) operating said last named node to continue its 
repeated scanning of said received TPN informa- 
tion, 

(2) comparing the results of successive scans, 

(3) determining when said received TPN information 
remains unchanged for a predetermined interval of 
time, and 

(4) analyzmg said received TPN information after the 
expiration of said interval to select as a preferred 
clock signal for said node the signal that has en- 
countered the least number of intermediate nodes 
in traveling from said master node to said receiving 
node. 

4. The method of claim 1 in combination with the 
additional steps of: 

(1) operating each node, other than said master node, 
to repeatedly scan said TPN information it re- 
ceives, 

(2) operating one of said other nodes as a backup 
master node, 

(3) operating said backup master node to detect when 
said received TPN information associated with 
said preferred clock signal changes from a prior 
value to a predetermined value, 

(4) switching the timing control of said backup mas- 
ter node from said preferred received clock signal 
to a signal from a local oscillator internal to said 
backup master node upon the detection of said 
change to said predetermined value, and 
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(5) transmitting a clock signal containing TPN infor- 
mation over directly connected ones of said links 
from said backup master node to control the timing 
of all nodes of said system other than said master 
node. 

5. The method of controlling the timing of each node 
in a multinode system whose nodes are interconnected 
by links and wherein a clock source in one of said nodes 
provides a reference clock signal source for all other 
nodes in said system, said method comprising the steps 
of: 

(1) operating one of said nodes as a master node to 
control the. timing of all nodes of said system, 

(2) including a timing priority number (TPN) in a 
clock signal transmitted from said master node 
over a direct link to at least another one of said 
nodes, 

(3) operating each node, other than master node, so 
that it alters a TPN it receives over a directly con- 
nected link in a prescribed manner to form a new 
TPN, 

(4) further operating each node, other than said mas- 
ter node, so that each said node sends a clock signal 
together with its altered TPN to all nodes to which 
it is directly connected by a link, 

(5) operating each node, other than said master node 
to identify one of the TPNs received as a preferred 
TPN based upon a prescribed relationship of said 
received TPNs, and 

(6) controlling the timing of said nodes, other than 
said master node, by the clock signal received over 
the link over which said preferred TPN is received 
by said node. 

6. The method of claim 5 in combination with the 
additional steps of: 

(1) operating each node, other than said master node, 
to repeatedly scan the TPNs it receives, 

(2) detecting a change when the received TPNs asso- 
ciated with said preferred clock signal changes on 
successive scans, and 

(3) switching the timing control of said node that 
receives said changed TPNs from said preferred 
clock signal to a signal from a local oscillator inter- 
nal to said last named node upon the detection of 
said change. 

7. The method of claim 6 in combination with the 
additional steps of: 

(1) operating said last named node to continue its 
repeated scanning of said received TPNs, 

(2) comparing the results of successive scans, 

(3) determining when said received TPNs remain 
unchanged for a predetermined interval of time, 
and 

(4) analyzing said received TPN information after the 
expiration of said interval to select as a preferred 
clock signal for said node the clock signal received 
over the link on which a TPN having a prescribed 
relationship to the TPNs of other links is received. 

8. The method of claim 5 in combination with the 
additional steps of: 

(1) operating each node, other than said master node, 
to repeatedly scan said received TPNs, 

(2) operating one of said other nodes as a backup 
master node, 

(3) operating said backup master node to detect when 
said received TPN associated with said preferred 
clock signal changes from a prior value to a prede- 
termined value, and 
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(4) switching the timing control of said backup mas- 
ter node form said preferred received clock signal 
to a signal from a local oscillator internal to said 
backup master node upon the detection of said 
change to said predetermined value, and 

(5) transmitting a clock signal contain a TPN over 
directly connected ones of said links from said 
backup master node to control the timing of all 
nodes of said system other than said master node. 

9. A method of controlling the timing of a clock 
signal receiving node in a multinode system whose 
nodes are interconnected by links and wherein a clock 
source in a master one of said nodes provides a refer- 
ence clock signal source for controlling the timing of all 
other nodes in said system, said method comprising the 
steps of: 

(1) receiving timing priority number (TPN) informa- 
tion together with a clock signal on each different 
link connected to said clock signal receiving node 
with said information representing the number of 
intermediate nodes each received clock signal has 
encoimtered in traveling from said master node to 
said clock signal receiving node, 

(2) operating said clock signal receiving node to ana- 
lyze the TPN information it receives on all links to 
which it is connected to identify as a preferred 
clock signal the clock signal that has encountered 
the least number of intermediate nodes in traveling 
from said master node to reach said node, 

(3) modifying the TPN from the link from which said 
preferred clock signal was selected, 

(4) transmitting the modified TPN on all links to 
which said clock signal receiving node is con- 
nected, and 

(5) operating said signal receiving node to select said 
preferred clock signal as a reference timing source 
for said signal receiving node. 

10. A method of controlling the timing of each node 
in a multinode system wherein said nodes are connected 

40 by links and wherein a clock source in a master one of 
said nodes provides a reference clock signal for all other 
nodes in said system, said method comprising the steps 
of: 

(1) interconnecting each of said nodes to one or more 
other ones of said nodes via said links, 

(2) transmitting a clock signal between each of said 
nodes directly interconnected by any one of said 
links, 

(3) activating one of said nodes to operate as a master 
node by providing a reference clock signal for 
controlling the timing of said system, 

(4) including a timing priority number (TPN) of n in 
the clock signal sent from said master node to all of 
said nodes directly connected to said master node 
by any one said hnks, 

(5) operating each node directly connected by a link 
to said master node and receiving said TPN of n to 
alter n by m to form a TPN of n-f-m characterizing 
each said directly connected receiving node, 

(6) further operating each node having a TPN of 
n-f m to include said TPN of n+m in the clock 
signal transmitted to each one of said nodes to 
which said node having a TPN of n-hm is directly 
connected by any one of said links, 

(7) operating all others of said nodes, other than said 
master node, that receives a TPN of n+m so that 
each of said other nodes alters a TPN it receives by 
m and transmits its altered TPN with a clock signal 
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to each node to which it is directly connected by 
any one of said Unks> 

(8) operating each node that receives more than a 
single TPN, other than said master node, to select 
one of said received TPNs as a preferred received 5 
TPN, and 

(9) controlling the timing of each node, other than 
said master node, from clock signals transmitted 
over the link that applies the preferred TPN to said 
controlled node. 10 

11. The method of claim 10 in combination with the 
additional steps of: 

(1) operating each liode, other than said master node, 
to repeatedly scan the TPNs it receives, 

(2) detecting a change when the received TPN asso- 15 
dated with said preferred clock signal changes on 
successive scans, and 

(3) switching the timing control of said node that 
detects a changed TPN from said preferred clock 
signal to a signal from a local oscillator internal to 20 
said last named node upon the detection of said 
change. 

12. The niethod of claim 11 in combination with the 
additional steps of: 

(1) operating said last named node to continue its 25 
repeated scanning of said received TPNs, 

(2) comparing the results of successive scans, 

(3) determining when said received TPNs remain 
unchanged for a predetermined interval of time, 
and 30 

(4) analyzing said received TPNs after said interval to 
select as a preferred clock signal for said node the 
clock signal received over the link on which a TPN 
having a prescribed relationship to the TPNs of 
other links is received. 35 

13. The method of claim 10 in combination with the 
additional steps of: 

(1) operating each node, other than said master node, 
to repeatedly scan said received TPNs, 

(2) operating one of said other nodes as a backup 40 
master node, 

(3) operating said backup master node to detect when 
said received TPN associated with said preferred 
clock signal changes from a prior value to a prede- 
termined value, and 45 

(4) switching the timing control of said backup mas- 
ter node from said preferred received clock signal 
to a signal from a local oscillator internal to said 
backup master node upon the detection of said 
change to said predetermined value, and 50 

(5) transmitting a clock signal containing a TPN over 
directly connected ones of said links from said 
backup master node to control the timing of all 
nodes of said system other than said master node. 

14. Facilities for controlling the timing of each node 55 
in a multinode system whose nodes are interconnected 
by links and wherein a clock source in one.of said nodes 
provides a reference clock signal source for controlling 
the timing of all other nodes in said system, said facili- 
ties comprising: 60 

means for operating one of said nodes as a master 
node to control the timing of all nodes of said sys- 
tem, 

means for transmitting a clock signal between each 
pair of said nodes directly interconnected by any 65 
one of said links, 

means for including timing priority number (TPN) 
information in the clock signal transmitted between 
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each pair of directly connected nodes with said 
information representing the priority of said clock 
signal, 

means for operating each clock signal receiving node, 
other than said master node, to analyze the TPN 
information it receives on all links to which it is 
connected to identify a preferred clock signal, 

means for modifying the TPN from the directly con- 
nected node which transmitted said preferred 
clock signal, 

means for transporting the modified TPN to all nodes 
directly connected to said clock signal receiving 
node, and 

means for operating each clock signal receiving node 
to select said preferred clock signal as a reference 
timing source for said receiving node. 

15. The facilities of claim 14 in combination with: . 
means for operating each node, other than said master 

node, to repeatedly scan said TPN information it 
receives, 

means for detecting when the received TPN informa- 
tion associated with said preferred clock signal 
changes on successive scans, and 

means for switching the timing control of said node 
that receives said changed TPN information from 
said preferred clock signal to a signal from a local 
oscillator internal to said last named node upon the 
detection of said change. 

16. The facilities of claim 15 in combination with: 
means for operating said last named node to continue 

its repeated scanning of said received TPN infor- 
mation, 

means for comparing the results of successive scans, 
means for determining when said received TPN in- 
formation remains unchanged for a predetermined 
interval of time, and 
means for analyzing said received TPN information 
after the expiration of said interval to select as a 
preferred clock signal for said node the signal that 
has encountered the least number of intermediate 
nodes in traveling from said master node to said 
receiving node. 

17. The facilities of claim 14 in combination with: 
means for operating each node, other than said master 

node, to repeatedly scan said TPN information it 
receives, 

means for operating one of said other nodes as a 
backup master node, 

means for operating said backup master node to de- 
tect when said received TPN information associ- 
ated with said preferred clock signal changes from 
a prior value to a predetermined value, 

means for switching the timing control of said backup 

, master node from said preferred received clock 
signal to a signal from a local oscillator internal to 
said backup master node upon the detection of said 
change to said predetermined value, and 

means for transmitting a clock signal containing TPN 
information over directly connected ones of said 
links from said backup master node to control the 
timing of all nodes of said system other than said 
master node. 

18. FacDities for controlling the timing of each node 
in a multinode system whose nodes are interconnected 
by links and wherein a clock source in one of said nodes 
provides a reference clock signal source for all other 
nodes in said system, said facilities comprising: 
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means for operating one of said nodes as a master 
node to control the timing of all nodes of said sys- 
tem, 

means for including a timing priority number (TPN) 
in the clock signal transmitted from said master 3 
node over a direct link to at least another one of 
said nodes, 

means for operating each node, other than master 
node, so that it alters a TPN it receives over a 
directly connected link in a prescribed manner to 10 
form a new TPN, 

means for further operating each node, other than 
said master node, so that each said node sends a 
clock signal together with its altered TPN to all 
nodes to which it is directly connected by a link, 15 

means for operating each node, other than said master 
node, to identify one of the TPNs received as a 
preferred TPN based upon a prescribed relation- 
ship of said received TPNs, and 

means for controlling the timing of each said node, 20 
other than said master node, by the clock signal 
received over he link over which said preferred 
TPN is received by said node. 

19. The facilities of claim 18 in combination with: 
means for operating each node, other than said master 25 

node, to repeatedly scan said TPNs it receives, 
means for detecting a change when the received TPN 
associated with said preferred clock signal changes 
on successive scans, and 
means for switching the timing control of said node 30 
that receives a changed TPN from said preferred 
clock signal to a signal from a local oscillator inter- 
nal to said node upon the detection of said change. 

20. The facilities of claim 19 in combination with: 
means for operating said last named node to continue 35 

its repeated scanning of said received TPNs, 
means for comparing the results of successive scans, 
means for determining when said received TPNs 
remain unchanged for a predetermmed interval of 
time, and 40 
means for analyzing said received TPNs to select as a 
preferred clock signal for said node the clock signal 
received over the link on which a TPN having a 
prescribed relationship to the TPNs of other links 
is received. 45 

21. The facilities of claim 18 in combination with; 
means for operating each node, other than said master 

node, to repeatedly scan said received TPNs, 

means for operating one of said other nodes as a 
backup master node, 50 

means for operating said backup master node to re- 
peatedly scan said received TPNs to detect when 
said received TPN associated with said preferred 
clock signal changes from a prior value to a prede- 
termined value, and 55 

means for switching the timing control of said backup 
master node from said preferred received clock 
signal to a signal from a local oscillator internal to 
said backup master node upon the detection of said 
change to said predetermined value, and 60 

means for transmitting a clock signal containing a 
TPN over directly connected ones of said links 
from said backup master node to control the timing 
of all nodes of said system other than said master 
node. 65 

22. Facilities for controlling the timing of each node 
in a multinode system wherein said nodes are connected 
by links and wherein a clock source in a master one of 
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said nodes provides a reference clock signal for all other 
nodes in said system, said facilities comprising: 
links for directly interconnecting each of said nodes 

to one or more other of said nodes, 
means for transmitting a clock signal between each of 
said nodes directly interconnected by any one of 
said links, 

means for activating one of said nodes to operate as a 
master node by providing a reference clock signal . 
for controlling the timing of said system, 

means for including a timing priority number (TPN) 
of n in the clock signal sent from said master node 
to all of said nodes directly connected to said mas- 
ter node by any one of said links, 

means for operating each node directly connected by 
a link to said master node and receiving said TPN 
of n to alter n by m to form a TPN of n+m charac- 
terizing each said directly connected receiving 
node, 

means for further operating each node having a TPN 
of n+m to include said TPN of n-|-m in the clock 
signal transmitted to each one of said nodes to 
which said node having a TPN of n-hm is directly 
connected by any one of said links, 

means for operating all others of said nodes, other 
than said master node, that receives a TPN of n-|-m 
so that each of sad other node alters a TPN it re- 
ceives by m and transmits its altered TPN with a 
clock signal to each node to which it is directly 
interconnected by any one of said links, 

means for operating each node that receives more 
than a single TPN, other than said master node, to 
select one of said received TPNs as a preferred 
received TPN, and 

means for controlling the timing of each node, other 
than said master node, from clock signals transmit- 
ted over the link that applies the preferred TPN to 
said controlled node. 

23. Facilities for controlling the timing of a clock 
signal receiving node in a multinode system whose 
nodes are interconnected by links and wherein a clock 
source in a master one of said nodes provides a refer- 
ence clock signal source for controlling the timing of alt 
other nodes in said system, said facilities comprising: 

means for receiving timing priority number (TPN) 
information together with a clock signal on each 
different link connected to said signal receiving 
node with said TPN information representing the 
number of intermediate nodes each received clock 
signal has encountered in traveling from said mas- 
ter node to said signal receiving node, 

means for operating said signal receiving node to 
analyze the TPN information it receives on all links 
to which it is connected to identify as a preferred 
clock signal the clock signal that has encountered 
the least number of intermediate nodes in traveling 
from said master node to reach said signal receiv- 
ing node, 

means for selecting said preferred clock signal as a 
reference timing source of said signal receiving 
node, 

means for modifying the TPN received from the link 
from which said preferred clock signal was re- 
ceived, and 

means for transmitting the modified TPN on all links 
to which said signal receiving node is connected. 

24. Facilities for controlling the timing of a clock 
signal receiving node in a multinode system whose 
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nodes are interconnected by links said facilities compris- 
ing: 

means for receiving timing priority number (TPN) 
information together with a clock signal on each 5 
different link connected to said signal receiving 
node with said TPN information representing the 
relative priority of each received clock signal^ 

means for operating said signal receiving node to 
analyze the TPN information it receives on all links 
to which it is connected to identify a preferred 
clock signal having a preferred priority, 

means for selecting said preferred clock signal as a 
reference timing source for said signal receiving 
node, 

means for modifying the TPN received from the link 
from which said preferred clock signal was re- 
ceived, and 20 
means for transmitting the modified TPN on all links 
to which said signal receiving node is connected. 
25. A method of controlling the timing of each nodes 
in a multinode system whose nodes are interconnected 23 
by links and wherein a clock source in one of said nodes 
provides a reference clock signal source for controlling 
the timing of all other nodes in said system, said method 
comprising the steps of: 
(1) operating one of said nodes as a master node to 
control the timing of all nodes of said system, 



(2) transmitting a clock signal between each pair of 
said nodes directly interconnected by any one of 
said links, 

(3) including timing priority number (TPN) informa- 
^ tion in the clock signal transmitted between each 

pair of directly connected nodes with said informa- 
tion representing the relative priority of the clock 
signal used by each transmitting node, 

(4) operating each signal receiving node, other than 
said master node, to analyze the TPN information 
it receives on all links to which it is connected to 
identify a preferred clock signal having a preferred 
priority, and 

(5) operating each clock signal receiving node, other 
than said master node, to select said preferred 
clock signal as a reference timing source for said 
receiving node, 

(6) modifying the TPN received from the link from 
which said preferred clock signal was received, 
and 

(7) transmitting the modified TPN on all links to 
which said receiving node is connected. 

26. The method of claim 25 in combination with the 
additional steps of: 

(1) detecting when the received TPN information 
associated with said preferred clock signal 
changes, and 

(2) switching the timing control of each node that 
detects a change from said preferred clock signal to 
another clock signal upon the detection of said 
change. 
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